Web Survey Bibliography
Background: Nonprobability Web surveys using volunteer panels can provide a relatively cheap and quick alternative to traditional health and epidemiological surveys. However, concerns have been raised about their representativeness.
Objective: The aim was to compare results from different Web panels with a population-based probability sample survey (n=8969 aged 18-44 years) that used computer-assisted self-interview (CASI) for sensitive behaviors, the third British National Survey of Sexual Attitudes and Lifestyles (Natsal-3).
Methods: Natsal-3 questions were included on 4 nonprobability Web panel surveys (n=2000 to 2099), 2 using basic quotas based on age and sex, and 2 using modified quotas based on additional variables related to key estimates. Results for sociodemographic characteristics were compared with external benchmarks and for sexual behaviors and opinions with Natsal-3. Odds ratios (ORs) were used to express differences between the benchmark data and each survey for each variable of interest. A summary measure of survey performance was the average absolute OR across variables. Another summary measure was the number of key estimates for which the survey differed significantly (at the 5% level) from the benchmarks.
Results: For sociodemographic variables, the Web surveys were less representative of the general population than Natsal-3. For example, for men, the average absolute OR for Natsal-3 was 1.14, whereas for the Web surveys the average absolute ORs ranged from 1.86 to 2.30. For all Web surveys, approximately two-thirds of the key estimates of sexual behaviors were different from Natsal-3 and the average absolute ORs ranged from 1.32 to 1.98. Differences were appreciable even for questions asked by CASI in Natsal-3. No single Web survey performed consistently better than any other did. Modified quotas slightly improved results for men, but not for women.
Conclusions: Consistent with studies from other countries on less sensitive topics, volunteer Web panels provided appreciably biased estimates. The differences seen with Natsal-3 CASI questions, where mode effects may be similar, suggest a selection bias in the Web surveys. The use of more complex quotas may lead to some improvement, but many estimates are still likely to differ. Volunteer Web panels are not recommended if accurate prevalence estimates for the general population are a key objective.
Web survey bibliography (4086)
- Do online access panels really need to allow and adapt surveys to mobile devices? ; 2014; Revilla, M.; Toninelli, D.; Ochoa, C.; Loewe, G.
- Why you need to make your surveys mobile friendly NOW; 2014; Lorch, J.; Mitchell, N.
- Assessing the Impact Device Choice Has on Web Survey Data Collection ; 2014; Hupp, A.; Schroeder, H. M.; Piskorowski, A.D.
- Understanding Mobility: Consent and Capture of Geolocation Data in Web Surveys; 2014; Crawford, S. D.; McClain, C.; Young, R.H.; Nelson, T. F.
- Swipe, Snap & Chat: Mobile Survey Data Collection Using Touch Question Types and Mobile OS Features ; 2014; Buskirk, T. D.; Michaud, J.; Saunders, T.
- Statistical Approaches to Analyze Self-Reported Susceptibility to Driver Distraction; 2014; Chen, H-Y. W.; Donmez, B.; Ko, Y-D.
- Using Web Panels for Official Statistics; 2014; Bethlehem, J.
- The problem of non-response in population surveys on the topic of HIV and sexuality: a comparative study...; 2014; Wallander, L.; H.; Mannheimer, L. N.; Oestergren, P. O.; Plantin, L.Tikkanen, R. H.
- Does the Length of Fielding Period Matter? Examining Response Scores of Early Versus Late Responders; 2014; Dyer Yount, N.; Lewis, T.; Lee, K.; Sigman, R.
- FocusVision 2014 Annual MR Technology Report; 2014; Macer, T., Wilson, S.
- When it comes to mobile respondent experience and data quality, survey design matters; 2014; Mitchell, N.
- The Changing Landscape of Technology and its Effect on Online Survey Data Collection; 2014; Mitchell, N.
- Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method, 4th Edition; 2014; Dillman, D. A., Smyth, J. D., Christian, L. M.
- The survey playbook: how to create the perfect survey. (Vol.1); 2014; Champagne, M. V.
- Do your own online surveys. DYI and self serve market research; 2014; Cary, N.
- The Influence of Answer Box Format on Response Behavior on List-Style Open-Ended Questions; 2014; Keusch, F.
- Nonprobability Web Surveys to Measure Sexual Behaviors and Attitudes in the General Population: A Comparison...; 2014; Erens, B.; Burkill, S.; Couper, M. P.; C., Clifton, S., Tanton, C., Phelps, A., Datta, J., Mercer,...
- Luteal-phase support in assisted reproduction treatment: real-life practices reported worldwide by an...; 2014; Vaisbuch, E., de Ziegler, D., Leong, M., Shoham, Z., Weissman, A.
- Facebook, Twitter, & Qr Codes: An Exploratory Trial Examining The Feasibility Of Social Media Mechanisms...; 2014; Gu, L. L.
- Time-dependent variation in the responses to the web-based ISAAC questionnaire; 2014; Yoshida, K., Sasaki, M., Odajima, H., Itazawa, T., Hashimoto, K., Furukawa, M., Adachi, Y.
- A Web-Based Survey To Evaluate The Factors Associated With Treatment Seeking Behavior In Adults With...; 2014; Bhattacharya, R., Dwibedi, N.
- 640 Current trends in management of high-risk prostate cancer in Europe: Results of a web-based survey...; 2014; Briganti, A., Isbarn, H., Ost, P., Ploussard, G., Sooriakumaran, P., Van Den Bergh, R.C.N., Van Oort...
- Perioperative management in order to minimise postoperative delirium and postoperative cognitive dysfunction...; 2014; Jildenstål, P. K., Jakobsson, J. G., Hallen, J. L., Berggren, L., Rawal, N.
- Data Collection Mode Effects On Political Knowledge; 2014; Liu, M., Wang, Y.
- Assessing the Potential of Paradata and Other Auxiliary Data for Nonresponse Adjustments; 2014; Krueger, B. S., West, B. T.
- QRTEngine: An easy solution for running online reaction time experiments using Qualtrics; 2014; Barnhoorn, J. S., Haasnoot, E., Bocanegra, B. R., van Steenbergen, H.
- Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis; 2014; Gnambs, T., Kaspar, K.
- Build your own social network laboratory with Social Lab: a tool for research in social media; 2014; Garaizar, P., Reips, U.-D.
- Is Vague Valid? The Comparative Predictive Validity of Vague Quantifiers and Numeric Response Options...; 2014; Al Baghal, T.
- The Effect of Answering in a Preferred Versus a Non-Preferred Survey Mode on Measurement; 2014; Smyth, J. D., Olson, K., Kasabian, A.
- Self-reported cheating in web surveys on political knowledge; 2014; Jensen, C., Thomsen, J. P. F.
- The invisible Interviewer: Old and New Methodological Issues in Online Research; 2014; Niero, M.
- Improving Survey Response Rates in Online Panels Effects of Low-Cost Incentives and Cost-Free Text Appeal...; 2014; Pedersen, M. J., Nielsen, C. V.
- Open-ended questions in Web Surveys-Using visual and adaptive questionnaire design to improve narrative...; 2014; Emde, M.
- Query on Data Collection for Social Surveys; 2014; Blanke, K., Luiten, A.
- A mixed-methods study on perceptions towards use of Rapid Ethical Assessment to improve informed consent...; 2014; Addissie, A., Davey, G., Newport, M. J., Addissie, T., MacGregor, H., Feleke, Y., Farsides, B.
- Questionnaire length and breakoffs in web surveys: a meta study; 2014; Vehovar, V., Cehovin, G.
- The need of and the demand for completing surveys on mobile devices; 2014; Toninelli, D., Revilla, M., Ochoa, C.
- Survey participation via mobile devices in a probability-based online-panel: Prevalence, determinants...; 2014; Poggio, T., Bosnjak, M., Weyandt, K.
- Matrix versus paging designs in a brand attribution task; 2014; Conrad, F. G., McCullough, W., Nishimura, R.
- Web-respondent-driven sampling; 2014; Bianchi, A., Biffignandi, S., Artaz, R.
- Off-Label Baclofen Prescribing Practices among French Alcohol Specialists: Results of a National Online...; 2014; Rolland, B., Paille, F., Fleury, B., Cottencin, O., Benyamina, A., Aubin, H.-J.
- Influences on regression testing strategies in agile software development environments; 2014; Parsons, D., Susnjak, T., Lange, M.
- The New Majority: How Will Latino Youth Succeed in the Context of Low Educational Expectations and Assumptions...; 2014; Gomez, C. A., Villasenor, E., Mann, E. S., Mandic, C. G., Valladares, E. S., Mercado, V., et al.
- The role of email addresses and email contact in encouraging web response in a mixed mode design ; 2014; Cernat, A., Lynn, P.
- Correlates of Individual Versus Joint Participation in Online Survey Research with Same-Sex Male Couples...; 2014; Millar, B. M., Parsons, J. T., Starks, T. J.
- Testing if Healthy Perfectionism Enhances Academic Achievement in Australian Secondary School Students...; 2014; Thorpe, E., Nettelbeck, T.
- Emotion management in online groupwork reported by Chinese students; 2014; Xu, J., Du, J., Fan, X.
- Examining the Effect of Prenotification Postcards on Online Survey Response Rate in a University Graduate...; 2014; Lalasz, C. B., Doane, M. J., Springer, V. A., Dahir, V. B.
- The Power of Partisanship in Brazil: Evidence from Survey Experiments; 2014; Samuels, D., Zucco, C.